Quantitative assessment of collaboration

This position paper presents a short literature review of a research trend that endeavors to model collaboration by quantifying each group member’s contribution. In such a view, equity is considered as the ideal collaborative situation. We review some foundational elements of this approach, some methodological aspects and examples of design implications for Computer-Supported Cooperative Work. 1 Equity as a paradigm for collaboration Our aim in this position paper is to present a research trend initially born in Psychology and Management science and later used in Human-Computer Interaction, modeling collaboration through quantification of each participant’s contributions. In this approach equity is sought, whatever the quality of contributions. Indeed for tasks involving negotiation, for collaborative learning, and every time it is important for all members to have their say, equity per se is a desirable state (Marshall et al., 2008) regardless of the quality of contributions. Disregarding the quality of contributions (at least at first) is also justified in the context of tasks such as design, where a strongly established paradigm points to “team idea generation” as a key element of work. The relevance of assessing collaboration based on individual contributions is further questioned by the fact that these contributions are merged together when constructing a solution to a design problem. Finally, equity does not need to yield visible “gains in performance” to be relevant in a collaborative work setting: beyond the fact that contributions from multiple participants are expected to speed up exploration of the problem space, ensuring that design decisions are made through integrating multiple points of view is a major stake for design. Therefore, a given quality of result is more desirable per se when obtained democratically (i.e. with higher equity) than when obtained by dominant leaders. Indeed, to Habermas (1984), “democracy” as opposed to a repressive communicational framework refers to a set of ways to ensure the information communicated by the various participants is done so with minimal distortion. When a tradeoff appears necessary between equity and efficiency, their respective weight will depend on the stakes and context of the task: in a negotiation context priority should be given to equity whereas in e.g. crisis management priority should be given to performance, be it obtained by a dominant leader. The equity paradigm has given rise to the observation of social phenomena such as social loafing and social compensation (Karau & Williams, 1993; Serva & Fuller, 1997): in a group situation, some participants tend to under-contribute with comparison to a situation where they would work alone (which is called social loafing) and other participants tend to over-contribute (social compensation). The occurrence of social loafers and social compensators results in the emergence of leaders and followers. It was shown that social loafing can be moderated by e.g. group cohesiveness (Karau & Hart, 1998), self-evaluation (Harkins & Szymanski, 1988), individual motivation (Brickner et al., 1986; Shepperd, 1993) or by the use of special collaborative devices as will be reported in section 3. 2 Methods for measuring equity in collaboration Several metrics have been proposed to measure the equity of collaboration: • The standard deviation of interface actions made by individuals (Ringel Morris et al., 2006): the larger the standard deviation, the less equitable the collaboration. A disadvantage of standard deviation is that it varies with both group size and the total number of actions, it is therefore difficult to compare across different study designs (Marshall et al., 2008). • The Gini Coefficient (Fitze, 2006) which has been used to measure the equity of contribution in groupware systems, classroom dialogue, economic income distributions, etc. It varies between 0 (perfect equality) and 1 (perfect inequality: 1 person has all of the income). However, the Gini coefficient in its standard form seems unsuitable for small numbers of participants (Marshall et al., 2008). • In a recent study about a brainstorming activity, we used the A index of inequality (see Table 1) where N=size of the group, E=the expected proportion of events if each participant contributes equally, and Oi=the observed proportion of events for each individual (Buisine et al., submitted). A normalized version of such index (see Table 1, Equation B) can also be used (Hiltz et al., 1989; Marshall et al., 2008) when one intends to compare varying tasks or contexts or study designs.